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Abstract 

Adjunct aids are instructional interventions inserted in textbooks in view of supporting 
learners to process the information. Different types of adjunct aids are assumed to support 
different cognitive processes. Research on adjunct aids has focussed on the learning effects of 
single types of adjunct aids. In this study, the learning effects of combinations of adjunct aids 
are studied. After completing a pre-test, 255 students studied a text presented on 21 separate 
computer screens in one of eight conditions. Their use of the adjunct aids was logged. After 
studying the text, a post-test was administered. Results show no impact of conditions on 
either knowledge or transfer items. However, when the use of the adjunct aids is considered, 
it is shown that this result can be explained by referring to the use of the aids at least for 
knowledge items. 


Introduction 

Adjunct aids are instructional interventions inserted in textbooks in view of supporting 
learners to process the information. Different types of adjunct aids are assumed to support 
different cognitive processes. Research on the effects of adjunct aids has flourished for the 
fourty years (for overviews, see: Rothkopf, 1996; Grabowski, 2004). This research has 
largely been part of the attempts to better understand how learners process text and how text 
comprehension can be promoted (e.g., Van Dijk & Kintsch, 1983). Having a cognitive 
psychological orientation, research on adjunct aids has contributed to elaborating 
information-processing models and to identifying implications of such models for learning 
and teaching. Results on retention and transfer tests, for instance, have been used as 
indicators of the kind of cognitive processing learners engage in (Mayer, 1996). Given its 
cognitive orientation, research on adjunct aids has been carried out in highly controlled 
settings. In most cases the learning effects of one particular type of adjunct aids and of fixed 
numbers of adjunct aids have been studied. Prototypical are the studies on advanced 
organizers (e.g., Mayer, 1979) or inserted questions (e.g., Andre, 1979). While this approach 
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may help to increase internal validity, it may at the same time reduce the ecological validity 
of these studies. The results may not apply to actual instructional settings in which designers 
and developers of instructional texts insert multiple and different types of adjunct aids in their 
materials. In view of increasing the ecological validity of research on adjunct aids, this study 
investigated the learning effects of combinations of adjunct aids. Furthermore, it is essential 
to study multiple aids in order to identify possible interaction effects. 

Research on adjunct aids in textbooks commonly assesses the learning effects by analysing 
the results of a post-test without analysing the actual use of the adjunct aid by the learners. 
This is a problem because multiple studies on instructional interventions indicate that learners 
often do not or not as intended use instructional interventions (Clarebout & Elen, in press). In 
order to deal with this problem, this study logged the students’ use of the adjunct aids when 
studying a text on a computer screen. 

The main question in this study addresses the impact of combinations and varying numbers of 
aids on the use and the learning effects of adjunct aids in textbooks. In addition, it is studied 
how the use of adjunct aids affects learning outcomes. 

In order to answer these questions, students were asked to study a text that was presented on a 
computer screen. In the experimental conditions (combinations of) three types of adjunct aids 
(questions, figures, and examples) were inserted in the text, whereas the text in the control 
condition contained no adjunct aids. Examples, questions and figures were selected because 
they clearly differ in nature and have been argued to trigger different cognitive functions. In 
line with the selecting, organizing, integrating (SOI) model, Mayer (1996, 1999) has 
suggested that different instructional interventions may support different categories of 
learning activities. Adjunct questions have been argued to support learners to focus on the 
relevant information (selecting); figures have been argued to support the organization of 
incoming materials (organizing), while examples as well as illustrations were suggested to 
help learners to activate prior knowledge and integrate new information in prior knowledge 
(integrating). Although these aids were added to have a specific mathemagenic or learning- 
generating effect (Rothkopf, 1970), the absence of a one-to-one relationship between an 
adjunct aid and a particular cognitive function is acknowledged (Kealy, Bakriwala, & 
Sheridan, 2003). 

This contribution discusses the empirical study after having briefly reviewed the research on 
the effects of the three aforementioned adjunct aids and the actual use of aids by learners. 


Adjunct Aids 


Questions 

Inserted questions have attracted ample research attention (e.g., Andre, 1987). This research 
shows that questions have mainly a selective and a restructuring function. Research 
commonly shows that readers are better able to recall topics targeted by pre-questions (Kealy 
et al., 2003). In the case of post-questions a broader generative effect was found. Post- 
questions may encourage readers to mentally review the material. 
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Figures 

In various studies the use of figures in addition to textual information has been proven to be 
highly valuable (for a review: Anglin, Vaez, & Cunningham, 2004). Mayer (2001) called this 
the multimedia effect. Levie and Lentz (1982) identified four functions for graphics/pictures 
in text: an attentional, an affective, a cognitive, and a compensatory function. While initially 
Paivio’s dual coding theory (Clark & Paivio, 1991) induced widespread optimism about the 
potential of pictures, recent research has induced more caution. In line with their integrated 
model of text and picture comprehension, Schnotz and Bannert (2003) for instance, 
concluded that only task-appropriate graphics may support learning, whereas task- 
inappropriate pictures or graphics may induce mathematantic effects. In the latter case a 
decrease rather than an increase in learning is observed. 


Examples 

Examples are so-called explanatory adjunct aids (Elen, 1995). They present additional 
information that may help the reader to better understand the text. Increased text 
comprehension results from helping the learner to connect the information in the text with the 
reader’s prior knowledge (Twohig, 1982). This is assumed to help the integration of the new 
information. While in general positive effects of examples can be expected, research shows 
that examples have to meet criteria such as understandability and relevance (Elen & Lowyck, 
2000). Furthermore, research on seductive details (Harp & Mayer, 1998) in which interesting 
but less relevant illustrations are added to the information, also reveals that adding highly 
interesting information for instance in examples, might be detrimental for learning. 


Use of Aids 

Research on adjunct aids has mainly been done in (quasi-) experimental settings. Various 
researchers have pointed out that the use of aids in such settings may differ from use in more 
ecological settings. More specifically, studies show that regularly adjunct aids are only 
seldom used as intended by the text designers. This is the case when instructional 
interventions such as adjunct aids are added to the information (Clarebout & Elen, in press) 
as well as when embedded instructional interventions are provided (Martens, Valcke, 
Poelmans, & Daal, 1996) 


Research Questions 

Considering the outcomes of previous research on adjunct aids, this study investigated the 
effect of combinations and varying numbers of adjunct aids on learning outcomes 
(knowledge and transfer). Because previous research explains the lack of effects by referring 
to the minimal use of adjunct aids, the use of the aids was also analysed. The relationship 
between the use of aids and their learning effect was studied. 

Research on combinations of adjunct aids is largely missing. Hence, there is only a limited 
basis for formulating hypotheses. Nevertheless, it is hypothesized that an effect of adjunct 
aids can be established and that the effect will be additive on the one hand and related to use 
of the aids on the other. The additivity hypothesis implies that an effect is expected for 
learning outcomes that are immediately related to a specific adjunct aid and that the overall 
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effects of adjunct aids become larger if more and different types of adjunct aids are made 
available. The use-hypothesis specifies that effects will only be found when the aids are 
actually used. 

In short, the study aims at answering two basic research questions. What are the learning 
effects of combinations and varying numbers of adjunct aids, and what is the relationship 
between the use of adjunct aids and learning outcomes? 


Method 


Participants and Design 

Research participants were 255 (221 female, 34 not specified) students attending 
undergraduate classes at a public university in the middle of South Africa. All students were 
majoring in education and volunteered to participate in the study. Students could gain a 
limited number of extra points by participating in the study. 

All respondents were randomly assigned to one out of eight conditions in a pre-test post-test 
experimental study with number of aids (five, ten or fifteen), type of aids (questions, 
examples or figures), and use of the aids as independent variables and learning outcomes 
(retention and transfer) as dependent variables. 

All students studied an instructional text on a computer screen. The eight conditions differ 
with respect to (a) the number of adjunct aids inserted (0, 5, 10, or 15 adjunct aids), and (b) 
the nature of the adjunct aids inserted (examples, questions, or figures) (see Table 1). In a 
first condition, the control condition, no adjunct aids were added. In conditions 2, 3, and 4, 
five adjunct aids were inserted: five examples in condition 2, five questions in condition 3 
and five figures in condition 4. The instructional text in conditions 5, 6 and 7 contained 10 
adjunct aids with in each condition two sets of five adjunct aids. In condition 5, 5 examples 
and 5 questions were inserted; in condition 6, 5 examples and 5 figures, and in condition 7, 5 
figures and 5 questions. Finally, in condition 8, 15 adjunct aids were inserted in the 
instructional text: 5 examples, 5 questions, and 5 figures. 


T able 1 Control and experimental conditions 



Examples 

Questions 

Figures 

N 

Condition 1 (control) 

0 

0 

0 

31 

Condition 2 

5 

0 

0 

31 

Condition 3 

0 

5 

0 

31 

Condition 4 

0 

0 

5 

31 

Condition 5 

5 

5 

0 

32 

Condition 6 

5 

0 

5 

32 
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Condition 7 

0 

5 

5 

32 

Condition 8 

5 

5 

5 

35 


Materials 

Instructional text 

The instructional text was a 6446 word passage (7182 words in the version with all three 
types of adjunct aids) adapted from a chapter on technology management by Newby, Stepich, 
Lehman and Russell (1996). The different versions of the computerised texts were developed 
using Macromedia Director 8.5. This text was selected by the teacher of this group of future 
teachers as being appropriate in level and also relevant given recent plans of the South- 
African department of education. Distributed over 21 screens, the text discusses various 
issues with respect to the integration of ICT in schools. In a first section the need for a 
technology plan is discussed. Hardware and software issues are discussed in the second and 
third section, while the final section describes personnel issues. On the last screen, a 
summary concludes the text. 
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Figure 1. Example of computer screen and related adjunct aid 
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Adjunct Aids 

Adjunct aids gained access to the instructional text by clicking on a labelled button. The 
integration of the adjunct aids is based on the following considerations. In order to ensure 
balance, adjunct aids were evenly spread over the 21 screens with a maximum of 1 adjunct 
aid per screen (e.g., two related screens are displayed in Figure 1). This means that in 
condition 8 with 15 aids, there were 15 computer screens with an aid and 6 screens without 
an aid. In order to ensure comparability of the eight conditions and to avoid interfering 
sequence effects, an adjunct aid was always linked to the same computer screen and different 
types of adjunct aids were inserted in the same sequence. These considerations resulted in a 
distribution of the adjunct aids as outlined in Table 2. 

Table 2 Distribution of adjunct aids over the text 
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Text screen 


1 2 3 4 5 6 7 

8 9 1 

1 1 

1 

1 

1 

1 1 

1 

1 

2 2 



0 

1 2 

3 

4 

5 

6 7 

8 

9 

0 1 

Questions 

X X 

X 


X 



X 




Figures 

X X 

X 



X 



X 



Examples 

X X 


X 



X 



X 



Questions are inserted in view of supporting information selection activities of students. 
Questions triggered students to think more deeply about an information element or to link a 
specific element to their person. In the conditions with inserted questions (conditions 3, 5, 7, 
and 8) students could access five inserted questions. The following question, for instance, 
was added to the screen in which different LAN-configurations were presented: “If your 
principal would ask your advice on the installation of a local-area-network (LAN), what 
would be your arguments in favour of a client-server model?”. 

Figure 2. Figure added to text explaining computer lab lay-outs. 



As suggested by the SOI-model, figures can have two main functions (Mayer, 1999). If a 
figure is more a graphic representation, organization processes are supported, while 
integrating processes may get supported when the main puipose of the figure is illustrative. In 
the conditions with figures (conditions 4, 6, 7, and 8) students could access five figures by 
clicking on the figure button. None of the figures was purely decorative. Three figures 
illustrated the information on the computer screen by repeating part of the textual information 
in a pictorial way (e.g., see Figure 2), whereas two others highlighted in a diagram the 
structure of the text on the screen that contained the figure button (e.g., see Figure 3). 

Figure 3. Figure added to screen discussing technology implementation plans. 
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Examples may support learners to connect their own prior knowledge with the new 
information. In the examples-conditions (conditions 2, 5, 6, and 8) students could access five 
examples that further contextualized the more general information in the text. Figure 1 shows 
an instance of an example in which a concrete software evaluation form is presented. The 
examples were constructed by selecting the most difficult element in the text. For instance, an 
excerpt from a mission statement was added to explain the meaning of the concept ‘mission 
statement’. 


Pre- and Post-test 

Two parallel tests were constructed. One was randomly selected and used as the pre-test. The 
other served as the post-test. Each test contained 8 items. Four questions were recall items. 
The information can be immediately retrieved in the text. The four remaining questions were 
transfer items. In order to generate an answer, students had to use and transform the 
information in the text. For instance, the text discusses considerations to be made while 
selecting hardware for classroom use. In the transfer item, it was asked what can go wrong 
when selecting hardware for classroom use. 

In order to analyze whether adjunct aids have a specific or a more general effect, some items 
directly related to an aid and others not. There was a clear relationship between one of the 
figures and two knowledge items in each test (see Figure 4), one example and one transfer 
item in each test, and one question and one transfer item in each test. For instance, in one of 
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the inserted questions students are encouraged to think about the implications of internet use 
although this is not explicitly discussed in the text. One transfer question addressed the use of 
the internet. 

Figure 4. Example of figure-related questions in pre- and post-test 

A. You see here a figure of a classroom arrangement. Circle the label that rightly describes 
this arrangement. 


1 . Peripheral 

2. Back-to-back 

3. Classroom 

4. Cluster 


^7 

XU 

XU 

XU 

/A 



u\ 

^7 

XU 

XU 

XU 



u\ 

ux 

Y7 

XU 

XU 

XU 

£7 

u\ 


n\ 


B. You see here a figure of a classroom arrangement. Circle the label that rightly describes 
this arrangement. 


l. 

Peripheral 

D-d 

D^d 

2. 

Back-to-back 



3. 

Classroom 



4. 

i 

Cluster 




Log-files 


All the actions of the participants were logged in an Access database for further analysis. The 
file contains an identification number for each respondent and information on whether and at 
what time a text screen or aid is accessed by the respondent. Data from the log files were 
used to calculate the number of times (specific types of) adjunct aids were used, the time 
spent on working with the aids and the proportion of the total study time devoted to the 
adjunct aids. Based on the data in the database different aspects of use could be calculated 
both for each adjunct aid separately and all aids inserted in the text: the number of times 
adjunct aids were accessed; the duration adjunct aids were looked at; and the proportion of 
the total studying time attention was devoted to adjunct aid(s). 


Procedure 

All participants were present in two sessions. The pre-test was administered during the first 
session. Respondents could take as much time as they needed to complete it. Administration 
of the pre-test took about ten to fifteen minutes. During the second session one week later, 
students in groups of maximum thirty at a time studied in a dedicated computer room the 
instructional text presented on an individual computer screen in one of the eight conditions. 
On an individual basis, participants were randomly assigned to one of the conditions (see 
Table 1). Each version presented participants with the instructional text and a different 
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combination of adjunct aids. After having studied the text and after having completed a non- 
related intermediate task in about 5 to 10 minutes, a post-test was administered containing 
items aiming at assessing knowledge and transfer. The total session took about 70 minutes. 
Five minutes were used for explaining the procedures during the session and how to handle 
the text on the screen. For this puipose a standardised PowerPoint presentation was shown. 
Students were told that they could study at their own pace and that they would be informed 
when 40 minutes had passed. After indicating that they had completed the study of the text, 
students received a non-related intermediate task. Completing the post-test took between 
fifteen and twenty minutes. 


Data Analyses 

As a first step in the analysis a paired t-test was done, to see whether students actually learned 
something. After having investigated the role of prior knowledge, the analysis focused on 
answering the two research questions. The first research question in this study relates to the 
impact of type and number of adjunct aid on learning outcomes. In order to answer this 
question one-way ANOVAs were done. Partial eta-squares were calculated to assess the 
effect-size. In line with Cohen (1988) an eta 2 -value between .01 and .06 is regarded as small 
to average, between .06 and .14 as average to large and above .14 as large to very large. The 
question on the impact of the use of aids on learning outcomes was also addressed though 
means of ANOVAs by taking the frequency of adjunct aids access, the absolute and relative 
time devoted to adjunct aids as independent variables and results on the post-test as 
dependent variables. For each indicator of use, three groups of students (low, middle, high) 
were created based on the percentiles. For analyses with respect to the use of adjunct aids, 
condition 1 (the control condition) was excluded given the absence of adjunct aids in this 
condition. 

All analyses were done with the statistical package SPSS and an alpha level of .05 was used 
as significance level for all statistical tests. In all cases DUNCAN was selected as the post- 
hoc test. 


Results 


Scoring 

Scoring of pre- and post test 

Pre-tests and post-tests were scored by awarding one point for each correct answer and one 
point for each relevant element in the open questions. The maximum number of points for the 
knowledge part in each test was 4 points. Given the way the transfer questions were scored, 
no maximum could be defined a priori. Table 3 gives an overview of the results. 

While the tests were conceptually completely parallel, paired samples t-tests reveal clear 
differences between results on the (different parts of) the pre- and post-test. 

Table 3: Results on pre- and post-test 

M SD M SD df t 
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Pre-test 



Post-test 





knowledge 

1.86 

1.00 

knowledge 

3.17 

.93 

220 

-16.096 

Pre-test 



Post-test 





transfer 

2.65 

1.95 

transfer 

4.81 

2.66 

220 

-11.551 

Pre-test total 

4.52 

2.35 

Post-test 

total 

7.98 

2.98 

220 

-17.33 


Scoring of log-files 

In order to get a good understanding of the use of the adjunct aids, the log-fdes were analysed 
to reveal the above mentioned indicators of the use of the adjunct aids: (a) the total number of 
times the adjunct aids were accessed, (b) the total number of times each specific type of 
adjunct aid was accessed, (c) the total time the adjunct aids were studied, (d) the time each 
specific type of adjunct aid was studied, (e) the proportion of the total study time spend on 
studying the adjunct aids, and (f) the proportion of the total study time spend on each type of 
adjunct aid. Table 4 summarises the main results of all the conditions. 

T able 4: Summary of log-file data 

Variable M SD 


Total access adjunct aids 

5.63 

5.32 

Total access questions 

1.61 

2.63 

Total access examples 

1.78 

2.40 

Total access figures 

2.23 

3.15 

Total time text 

36’02” 

9’4” 

Total time adjunct aids 

1’43” 

1 ’48' 

Total time questions 

0’30” 

0’48 

Total time examples 

0’50” 

1 ’23 

Total time figures 

0’22” 

0’32 

Proportion adjunct aids / total time 

4.65 

4.51 

Proportion questions / total time 

1.43 

2.20 

Proportion examples / total time 

2.15 

3.34 

Proportion figures / total time 

1.07 

1.60 
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Learning outcomes 

The initial analysis of the data has already revealed a clear difference between the results on 
the (different parts of the) pre- and the post-test. As a first step, the impact of prior 
knowledge on learning outcomes was investigated through means of one-way ANOVAs. 
Significant effects (see Table 5) with average to large effect-sizes were found for results on 
the pre-test as a whole, and on both the knowledge part and the transfer part. For the test as a 
whole a DUNCAN post-hoc test indicates that students with limited prior knowledge perform 
worse than students in the middle and high performing groups, whereas students in the 
middle group do also perform worse than students scoring high on the pre-test. For the 
knowledge part, DUNCAN highlights that students with low scores on the pre-test perfonn 
worse on the post-test than students from the middle- or high-scoring group. For the transfer 
part, a DUNCAN post-hoc test reveals the low performing group to score worse than the 
group with high scores on the pre-test. For the knowledge test, only marginally significant 
results were found. 


Table 5: Impact of pre-test results on post-test results 


Post-test 






Pre-test low 

Pre-test middle 

Pre-test high 

df 

F 

Partial 

Eta 2 

M 

SD 

M 

SD 

M 

SD 

Total 

2; 220 

17.34 

0.14 

6.55 

2.80 

8.27 

2.77 

9.26 

2.76 

Knowledge 

2; 220 

4.05 

0.04 

2.94 

.97 

3.26 

.92 

3.35 

.82 

Transfer 

2; 220 

10.55 

0.09 

3.91 

2.78 

4.68 

2.22 

5.94 

2.72 


The first actual research question pertains to the effect of different conditions on the learning 
outcomes. ANOVAs with condition as independent and the different aspect of the learning 
outcomes as dependent variables, reveal no effect of condition; for neither the post-test as a 
whole nor the knowledge- and transfer-parts separately. Only for two specific and similar 
items (items B and C: see Figure 4) were effects with average effect sizes were detected. 
Both items were linked to one specific aid, namely a picture that graphically clarified the 
concepts. A DUNCAN post-hoc test for the effect on the B item (F= (2; 248)= 2.141; partial 
eta 2 : .06) reveals a difference between the questions condition on the one hand and the 
figures-questions, the figures-examples-questions, the figures-examples, and the figures 
conditions on the other. A DUNCAN post-hoc test for the effect on the C item (F(7; 254) = 
2.074; eta 2 : .06) again reveals lower results for students in the questions-condition than for 
those in the figures- and the figures-examples conditions. Moreover, an additional difference 
was found between the examples-questions condition and the no-aids condition on the one 
hand and the figures-examples condition on the other. 

The impact of each adjunct aid was further investigated by looking for effects of respectively 
examples and no-examples conditions, questions and no-questions conditions and figures and 
no-figures conditions on related items. This may help to see whether the effect is specific or 
not. The analysis revealed an impact for the B item of the figures conditions (F(l; 224) = 
6.110; eta 2 : .03) indicating a better result of the figures conditions over the no-figures 
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conditions. For the C- item, an effect is found for the questions-conditions (F(l; 223) = 6.363; 
eta 2 : .03) showing that students in the no-questions conditions perform better than those in the 
questions-conditions. In both cases, effect sizes are rather small. 

A similar analysis was done to see whether the number of adjunct aids (5, 10 or 15) affected 
the learning outcomes. No impact of number of aids on learning outcomes was found. 

Use of adjunct aids 

A second research question pertains to the relationship between the use of the adjunct aids 
and the learning outcomes. For all three independent variables, effects with rather small 
effect sizes were found on the knowledge questions a. For the effect of the access of adjunct 
aids (F (2; 205) = 3.561; eta 2 : .04) the DUNCAN post-hoc test reveals a better result on the 
knowledge part for those students that used the aids most frequently in contrast to students 
who used them least frequently. Exactly the same results are found for the effect of 
proportion of time devoted to the adjunct aids (F(2; 205) = 3.879; eta 2 : .04). For the effect of 
the time spent on the adjunct aids (F(2; 205) = 4.529; eta 2 : .04) a DUNCAN post hoc reveals 
a difference between the students that spent most time and both the lowest and the middle 
group. 

Finally it was found that the total study time did not affect learning outcomes. 


Discussion 

The results clearly show that students learned while studying the text. Their performance on 
the different parts of the post-test is significantly higher than their perfonnance on the 
different parts of the pre-test. Furthermore, an effect of the pre-test could be revealed, which 
of course simply reconfirms the importance of prior knowledge. 

In spite of obvious learning outcomes, the hypotheses themselves were not validated. 
Inserting more adjunct aids does not result in better learning outcomes. Moreover, increased 
use of adjunct aids only has a limited effect on the knowledge part of the post-test. 

With respect to the impact of particular types of adjunct aids, results are less equivocal. 
Overall, an impact of adjunct aids is observed only in very specific cases. The inclusion of 
examples does not appear to contribute to learning. This might be due to the nature of the 
examples included. They intentionally illustrated the content in the text. For instance, the text 
argues for the need for a clear evaluation sheet for software evaluation. The related example 
presents such a sheet. For the questions conditions, an effect is found for only one specific 
item (the C item in figure 1). To attribute this to the specificity effect reported in the literature 
is a possibility. Its limited occurrence, however, calls for necessary caution. Similarly, an 
effect of figures is found for only one item (the B-item in figure 1). Clearly the aid may have 
helped simply because it contained the same figure that was used in the post-test. 

The overall absence of an impact of adjunct aids is further confirmed by the absence of any 
difference for conditions with different numbers of aids. In this respect, however, further 
research must consider the impact of the amount of aids independent from the study of 
interactions between types of aids. In this study both variables are not completely 
independent as more aids also implies that combinations of aids were inserted. 
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It must to be concluded that the hypotheses proved wrong. There is to some extent a specific 
effect of adjunct aids but overall there is no effect and certainly no additivity effect. Such 
results are not uncommon and a lack of such an effect is commonly attributed to under-use or 
inadequate use of the aids by the learners (e.g., Peeck, 1993). That is exactly the reason why 
in this study the actual use of the aids was logged. In this respect, results show a clear effect 
of the use of adjunct aids on the knowledge items in the post-test. In other words, students 
who accessed the aids more frequently, spent more time consulting the aids and devoted 
relatively more study time to them, did better on the knowledge items. This is in line with 
expectations., Adjunct aids can have an effect, only when actually used by the students. 
However, the results show no such effect for the transfer items. Three reasons may account 
for this lack of effect. First transfer questions might have been far too difficult. Second 
adjunct aids may have been insufficiently functional to promote transfer. In terms of the SOI- 
rnodel of Mayer, adjunct aids may have promoted selection and organisation but not 
integration. Of course, an adjunct aid can only promote transfer when functional in that 
respect. And third, the literature clearly shows that transfer-related learning outcomes require 
sufficient study time. It might be that generally, students simply did not study long enough to 
be able to answer the transfer questions. 


Conclusions 

In this study an attempt was made to acquire better insights in the interaction between adjunct 
aids. It was hypothesised that effects of adjunct aids would be additive and related to their 
use. In other words it was expected that adding adjunct aids would improve learning 
outcomes while recognizing that the use of the aids is a condition for any effect. This study 
hardly provides data that support these hypotheses. The study clearly reveals that students 
with higher prior knowledge do better on the post-test. It also indicates that students hardly 
use the aids. An effect of the aids is only found when they are actually used. Finally, the 
study seems to suggest that there is an effect of adjunct aids only when the information in the 
adjunct aids is directly functional to answering and item in the post-test. Although much care 
was taken in this study to ensure relevance of the adjunct aids, it is clear that in further 
studies the relationship between the adjunct aids and the post-test needs even more attention. 
More specifically, an attempt should be made to support all processes of the SOI-model 
(Mayer, 1996, 1999). This study induces the conclusions that adjunct aids are only beneficial 
if adapted to the learner, used by the learner, relevant to the task, and transparently linked to 
specific items in the post-test. 

The results of this study can be interpreted in different ways. First, it reconfirms the 
importance of prior knowledge. Second, it reconfirms the need for calibration between the 
learner on the one hand, and the (textbook) designer on the other in order for the learning 
environment to be effective (Winne, in press). It is up to the designer to ensure that the 
adjunct aid is linked to the task and to the learner; it is up to the learner to grasp the 
functionality of the aid and use it appropriately. A well-designed adjunct aid may have 
mathemagenic effects as Rothkopf has repeatedly argued (1970, 1996). However, this study 
illustrates that such a learning-generating effect requires learners to use the aids. The study 
also suggest that adjunct aids may have mathemantic or counter-productive effects (Clark, 
1988) if inadequately used by the learner. A well-designed aid is not sufficient; it needs to be 
adequately used by a knowledgeable learner. This reveals a clear limitation of this study. 
Further research on the use and effects of adjunct aids should not only register quantitative 
but also qualitative aspects of their use. Furthermore, future research should attempt to assess 
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how knowledgeable learners are about instructional interventions (Lowyck, Elen, & 
Clarebout, in press). 

Given all these results, the need for a renewed research effort on the effect of combinations of 
instructional interventions is clear. More particularly, in-depth investigations are needed on 
factors that determine whether and how such interventions are used. 
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